Papers with iterative NAR models
Isotropy-Enhanced Conditional Masked Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive models with auto-regressing decoding paradigms have been used for various text generation tasks to accelerate inference but at the cost of generation quality to some extent. |
| Approach: | They propose to use Look Neighbors strategy to enhance learning of target token representations during training to achieve a good balance between inference speedup and generation quality. |
| Outcome: | The proposed models outperform current models on 4 WMT datasets and outperformed the current SoTA results. |
An Empirical Study of Iterative Refinements for Non-autoregressive Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Iterative non-autoregressive (NAR) models have recently demonstrated impressive performance in varied generation tasks, surpassing the autoregressive Transformer. |
| Approach: | They propose a strategy to conduct efficient refinements without performance declines by using two simple metrics to identify potential problems existing in current refinement processes. |
| Outcome: | The proposed model outperforms the autoregressive Transformer by around one BLEU on average. |